A Wrapper Generation Toolkit to Specify and Construct Wrappers for Web Accessible Data Sources (websources)

There is an increase in the number of data sources that can be queried across the WWW. Such sources typically support HTML forms-based interfaces and search engines query collections of suitably indexed data. The data is displayed via a browser. One drawback to these sources is that there is no standard programming interface suitable for applications to submit queries. Second, the output (answer to a query) is not well structured. Structured objects have to be extracted from the HTML documents which contain irrelevant data and which may be volatile. Third, domain knowledge about the data source is also embedded in HTML documents and must be extracted. To solve these problems, we present technology to deene and generate wrappers for Web accessible sources (WebSources). Our contributions are as follows: (1) Deening a wrapper interface to specify the capability of WebSources. (2) Developing a wrapper generation toolkit of graphical interfaces and speciication languages to specify the capability of sources and the functionality of the wrapper. The toolkit provides a graphical interface to specify the capabilities of the sources and to deene a simple query translation and answer extraction process. It supports a language to specify a URLConstructor expression, for some query. It supports a declarative Qualiied-path-expression Extractor Language, QEL, to describe a simple Extractor that can extract data from a single HTML document. The toolkit also supports a Complex Extractor Speciication Language, CESL to specify extractors with more complex capability. The third contribution is (3) Developing the technology to generate a wrapper appropriate to the WebSource, from the speciications.